Generating composite or panoramic images, or more simply panoramas, from sets of still images or sequences of video frames (collectively “frames”) is known. In this manner, information relating to the same physical scene at a plurality of different time instants, viewpoints, fields of view, resolutions, and the like from the frames is melded to form a single wider angle image.
To generate a panorama, the various frames are geometrically and calorimetrically registered, aligned and then merged or stitched together to form a view of the scene as a single coherent image. During registration, each frame is analyzed to determine if it can be matched with previous frames. A displacement field that represents the offset between the frames is determined and then the frame is warped to the others to remove or minimize the offset.
In order for the panorama to be coherent, points in the panorama must be in one-to-one correspondence with points in the scene. Accordingly, given a reference coordinate system on a surface to which the frames are warped and combined, it is necessary to determine the exact spatial mapping between points in the reference coordinate system and pixels of each frame.
Prior to stitching, the frames to be used to form the panorama are typically arranged in the desired order (i.e. layout) thereby to facilitate the frame registration process. If the number of frames is small, manually performing this step is relatively easy. However, when the number of frames to be used to form the panorama is large, the task of ordering the frames can be time consuming and tedious. In such cases, being able to automatically estimate the frame layout is advantageous.
Techniques to mosaic images to form panoramas have been considered. For example, U.S. Pat. No. 6,078,701 to Hsu et al. discloses a method and apparatus for determining the topology of a sequence of images and then globally aligning the images with respect to each image's neighboring images. The topology determination and global alignment processes are iterated to progressively produce accurately aligned images allowing a plurality of source images to be combined into a seamless panoramic mosaic representation of a scene. During topology determination, pairs of images that spatially overlap each other are identified. Local coarse registration is performed to estimate a low complexity approximate spatial mapping between the neighboring images. Local fine registration is used to estimate a higher complexity mapping between neighboring images.
U.S. Pat. No. 6,798,923 to Hsieh to el. discloses a method of merging a pair of images to form a panoramic image. A set of feature points defining edge orientations are extracted along the edges of the images. Registration parameters are obtained by determining an initial set of feature points from an image which matches a set of feature points from another image. The registration parameters are then used to render the panoramic image.
U.S. Patent Application Publication No. US2004/0091171 to Bone discloses a method for constructing a mosaic image from an MPEG video sequence. The pictures of the MPEG compressed video are aligned and composited using motion vectors in the video sequence to achieve picture-to-picture registration.
Although the aforementioned references disclose different techniques for generating panoramas, there exists a need for automatically estimating the layout of frames to be used to generate a panorama. It is therefore an object of the present invention to provide a novel method, apparatus and computer program for automatically estimating the layout of a sequentially ordered series of frames to be used to form a panorama.